Skip to content

fix(kiro): estimate chat-file input tokens from every human turn, not a 500-char slice - #934

Open
ozymandiashh wants to merge 1 commit into
getagentseal:feat/core-extractionfrom
ozymandiashh:fix/port-kiro
Open

fix(kiro): estimate chat-file input tokens from every human turn, not a 500-char slice#934
ozymandiashh wants to merge 1 commit into
getagentseal:feat/core-extractionfrom
ozymandiashh:fix/port-kiro

Conversation

@ozymandiashh

Copy link
Copy Markdown
Collaborator

Ports upstream 6c4645a.

decodeKiroChatFile estimated input tokens from pendingUserMessage — the last human turn, sliced to 500 characters — while output summed every bot character. So any multi-turn session, or any final prompt over 500 characters, under-reported input tokens and therefore costUSD, severalfold in the worst case.

It now accumulates every human turn's full character count, matching the modern-execution, ws-session, CLI-session and V2 arms which already did. The 500-char cap stays for the display userMessage only.

Scope

The chat-file arm alone. The IDE-file dispatcher routes any record carrying a chat array plus metadata to decodeKiroChatFile, so this is chat-shaped Kiro IDE files — not every Kiro prompt. Identity preambles and tool content are excluded from the sum, so "full prompt" means the sum of human message characters rather than everything in the file.

The parity golden was defending the bug

kiro-golden.test.ts's G2 case was titled "A1 input tokens derived from truncated prompt" — it pinned the buggy value, because the corpus was captured before upstream's fix. Fix the decoder and it goes red.

The pin moves from 125 to 750, derived from the fixture rather than copied from the new output: the prompt is 3000 characters, the estimator is ceil(chars / 4), so 3000/4 = 750 where 125 was 500/4. A comment marks 125 as a pre-fix value so nobody restores it. No other kiro golden encoded the same assumption — checked.

Cache invalidation, and what this PR deliberately does not do

The user-visible part is cached history. Without invalidation, already-parsed sessions keep pre-fix tokens and cost forever while new ones use corrected maths — the report silently disagrees with itself.

The kiro provider parse fingerprint is bumped here, which discards the provider section and forces a re-parse of unchanged chat files. Verified independently: that check runs per provider section at scan time and has nothing to do with the daily cache.

The daily rollup also needs invalidating for the fix to be fully visible — days already finalized are returned as-is unless a savings-config change, a TZ change, a completeness flag or a gap fires a re-derivation. This PR does not bump that constant: a single bump in the release covers every parser-behaviour change landing with it, and #926 carries it. Three PRs editing the same line would conflict and leave you asking which "16" is which.

If this and #926 land apart, the effect is precise: the session cache re-parses so new numbers are correct, but a finalized daily rollup keeps pre-fix kiro cost until something else forces re-hydration.

Verification

Core 509, kiro 77, cache suites 67 — including a regression test that pins 125 under the old fingerprint and 750 under the new one.

@ozymandiashh
ozymandiashh marked this pull request as draft August 5, 2026 10:04
@ozymandiashh

Copy link
Copy Markdown
Collaborator Author

Converted to draft: this had two independent reviews but a third never landed. Re-running it; will mark ready once complete.

@ozymandiashh
ozymandiashh force-pushed the fix/port-kiro branch 3 times, most recently from a11876c to 35e4e42 Compare August 5, 2026 13:39
@ozymandiashh
ozymandiashh marked this pull request as ready for review August 5, 2026 13:39
@ozymandiashh
ozymandiashh marked this pull request as draft August 5, 2026 14:13
… invalidate cached history

decodeKiroChatFile estimated input tokens from pendingUserMessage - the last
human turn sliced to 500 chars - while output summed every bot char, so any
multi-turn session or final prompt over 500 chars under-reported input tokens
(and therefore costUSD) severalfold.

The estimate is now the sum of every human turn's full character count, with
the 500-char cap kept for the display userMessage only. That closes most of
the gap but does NOT reach parity with the modern-execution, CLI-session and
V2 arms: those count tool and system records as input - their code comments
state tool results are fed back to the model - while the chat arm still
counts only human records. Tool content demonstrably exists in the format
(the G2 fixture carries a tool record), so the chat arm still under-reports;
it just under-reports far less than before. The blast radius is the
chat-file arm alone: the IDE-file dispatcher routes any record carrying a
chat array plus metadata to decodeKiroChatFile, so this is chat-shaped Kiro
IDE files, not every Kiro prompt.

The identity-preamble exclusion now trims leading whitespace before its
startsWith match. Pre-fix, a near miss (a leading newline, a BOM, a wrapper)
was nearly harmless, because the preamble only mattered if it happened to be
the last human turn; post-fix, every unmatched system-injected human record
adds its FULL length to input, and preambles are large - a missed match is a
silent multi-thousand-token inflation on every affected session. Leading
whitespace tolerance is cheap (a genuine prompt never starts with whitespace
plus an identity tag) and the failure asymmetry favours exclusion: a false
negative inflates tokens, a false positive only skips a preamble. A renamed
preamble remains a residual risk, noted in the near-miss regression test.

Cached history is affected, which is what a user actually sees. session-cache
serves unchanged files without invoking the provider parser, so bump kiro's
PROVIDER_PARSE_VERSIONS fingerprint (ide-parsing-v1 -> v2) to force one
re-parse of every already-cached kiro session; without it the pre-fix token
and cost numbers would be served forever.

The daily rollup ALSO needs invalidating for this fix to be fully visible:
days finalized before the fix keep their pre-fix kiro cost in the daily
cache, and ensureCacheHydrated re-derives them only on a version bump, a
savings-config change, a timezone change, or an incomplete cache — the
session-cache re-parse alone leaves finalized day totals untouched. So this
commit bumps BOTH layers: the session-cache PROVIDER_PARSE_VERSIONS
fingerprint above forces the one re-parse of every already-cached kiro
session, and DAILY_CACHE_VERSION (15 -> 17, MIN_SUPPORTED_VERSION raised
with it; 16 is skipped because main already claimed it for the codex
structural-discovery fix, and claiming 16 here would load a main-built v16
cache as current and complete, so the invalidation would never fire) forces
the daily rollup's one-time re-derivation, so finalized day
totals are rebuilt under the corrected estimate. The re-derive reaches every
day whose kiro chat files still exist; sourceless days carry forward with
their pre-fix totals under the v14 NEVER-LOSE rule (a carry-forward, not a
refresh — nothing can reconstruct them once the files are gone).

Update the G2 parity golden: A1 was pinned at 125 tokens for a 3000-char
prompt (the 500-char slice / 4); the corrected value is 750 (3000 / 4), with
a comment marking 125 as a pre-fix value so it is not restored. 3000, 2400
and 1000+1000 are all exact multiples of four, so add G2b pinning the
estimator's rounding with an odd length (3001 chars -> 751 tokens; round and
floor would both give 750). Add a money-path regression test (2400-char
prompt -> 600 tokens, userMessage still 500-capped for display), a
multi-turn accumulation test (an identical resubmitted prompt counts again -
a real second model input; identity messages stay excluded), a near-miss
identity test (leading-newline and BOM preambles stay excluded), extend the
kiro cache-invalidation test to pin the v1 -> v2 fingerprint bump, and add a
daily-cache regression test seeding a complete pre-fix v15 cache (unchanged
savings hash and timezone, so nothing but the version bump can invalidate
it) and proving the bump forces the re-derive that lands the corrected kiro
cost — while the v15 file is never rewritten.
@ozymandiashh
ozymandiashh marked this pull request as ready for review August 5, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant